Skip to content

Harden namespace directory ownership and lifecycle cleanup - #36

Open
raphaelfeitoza wants to merge 13 commits into
fix/namespace-path-traversal-89265from
fix/namespace-directory-ownership-89265
Open

raphaelfeitoza wants to merge 13 commits into
fix/namespace-path-traversal-89265from
fix/namespace-directory-ownership-89265

Conversation

@raphaelfeitoza

@raphaelfeitoza raphaelfeitoza commented Sep 30, 2026 •

Copy link
Copy Markdown

Stack

Depends on #31 (fix/namespace-path-traversal-89265). Review this PR against that branch, not main; merge #31 first.

This extracts directory-ownership and lifecycle hardening from #31; its minimal namespace-string traversal fix remains independently reviewable.

Summary

  • Atomically reserve namespace directories and verify filesystem identity and exact-name ownership across create, fork, reset, load, and teardown.
  • Replace configurator delete-by-path cleanup with store-coordinated, identity-checked cleanup. Quarantine incompatible replica logs and unsafe-to-reuse directories; count retained cancellation/replica quarantines.
  • Coordinate operations with per-name locks rather than holding the global filesystem lock across remote work. Drain operations on bounded shutdown; distinguish internal default initialization from explicit creation.
  • Fence stale metastore handles and queued writes with incarnation tokens, avoiding deleted namespace/shared-schema link resurrection. Recheck primary existence on cache misses.
  • After backup confirmation, transfer destroy to a cancellation-independent worker. Persist a durable, atomically published per-name destroy intent before deleting SQLite metadata; on restart, reconcile metadata and the original directory/quarantine by filesystem identity before serving. Prevent reuse while the intent remains.
  • Transfer reset to an independently draining worker and retain the old directory through replacement setup. A durable pending/committed reset record preserves the old config/inode; incomplete setup fails closed and startup either restores the old incarnation or finishes a committed reset. Coordinate linked-schema migration registration with reset to avoid invalid migration membership and scheduler exhaustion. Replica reset skips the primary-only migration jobs table.
  • Add deterministic ownership, cancellation, SQL error, restart-reconciliation, migration-locking, default creation, replica reset and availability regressions.

Review and threat model

Independent correctness, security and read-only cross-file reviews found no remaining high/critical issue on final commit 744ca2f75f5cce3141b6400c62ee49fbc4cd1dbb within the trusted single-process/process-crash model. A Linux CI replica-reset regression on the preceding commit was fixed by 744ca2f75f and the same cluster test passed in the full rerun.

This is not a general atomic-filesystem or multi-writer security guarantee. Privileged external filesystem replacement is out of scope. Failed/incomplete resets and quarantined directories may require operator inspection; see docs/ADMIN_API.md. Sudden power-loss durability on Windows and device-level macOS F_FULLFSYNC are not claimed. Synchronous short filesystem operations under identity coordination avoid cancellation races but can block a Tokio worker on slow filesystems. Logical reopen tests do not substitute for actual SIGKILL/power-loss tests.

Validation

  • cargo fmt --all -- --check, cargo check -p libsql-server --tests --offline, git diff --check: passed.
  • Full Rust CI rerun on 744ca2f75f passed: both Run Tests jobs, Run Checks, unused-dependency/features check, Windows checks. The first run found a replica-only jobs table regression, fixed in this commit; the next run hit a DatabaseBusy failure in an unchanged concurrent-connection test, passed on retry; a subsequent full rerun passed.
  • C and Go bindings checks passed on this commit.
  • The separate Extensions Tests workflow fails before tests: runner Rust 1.85 versus transitive dependencies requiring Rust 1.88, also observed before this PR's latest changes. This is an existing CI/toolchain issue, not a test assertion.
  • Native macOS server tests cannot link locally because of unresolved libsql_open_v3 / sqlite3_* symbols; Linux CI supplies runtime validation.

penberg and others added 3 commits March 25, 2026 10:15
vectorParseSqliteText stores each vector element in a 1025-byte stack
buffer whose final byte must remain NUL. The length guard used `>`, so a
1025-character element overwrote that terminator before the guard
fired; the following error path then formatted the buffer with `%s`,
reading past the end of the stack buffer.

Reject the element once it reaches MAX_FLOAT_CHAR_SZ characters so the
terminator is preserved, regenerate the bundled amalgamations, and add
boundary regression tests for every vector function that parses TEXT.
@raphaelfeitoza
raphaelfeitoza added this pull request to stack #37 October 1, 2026 14:49
@raphaelfeitoza
raphaelfeitoza marked this pull request as ready for review October 1, 2026 18:29
@raphaelfeitoza
raphaelfeitoza requested a review from a team October 1, 2026 18:29
AryanSuvarna and others added 10 commits October 1, 2026 15:12
Fix out-of-bounds read when formatting overlong vector text elements
libsql-sqlite3/test/rust_suite has no committed Cargo.lock, so CI resolves
its transitive dependencies fresh on every run. Several of those now
require a newer compiler than the pinned 1.85.0 (icu_* and wasm-encoder/wast
declare rust-version 1.88; yoke-derive 0.8.3 declares none but uses
str::from_utf8 as an inherent method, stabilized in 1.87). This breaks the
Extensions Tests job and the rusttestwasm step of make-sqlite3 on main.

Pin current stable (1.98.1) rather than the minimum that compiles today
(1.88.0, verified green in CI), so crate MSRV bumps do not break CI again
in the near term. Document the constraint next to the unlocked test crate.
CI compiles with RUSTFLAGS="-D warnings", so lints added since 1.85.0
fail the build:

- mismatched_lifetime_syntaxes (new in 1.89): eleven signatures elide a
  lifetime on the input side (&self / &str) but hide it on the output type
  (Vec<Column>, PageHdrIter, CursorStep<S>, Cow<str>). Spell the output
  lifetime as '_ as the compiler suggests. No semantic change; this is the
  lifetime rustc already inferred.
- unused_assignments: `frameno` in bottomless-cli's restore loop was only
  ever copied into BatchReader::new and then incremented, never read.
  BatchReader tracks its own next_frame_no and the function returns the
  separate last_received_frame_no, so the local was dead since it was
  introduced in 4a71b20. Remove it and pass first_frame_no directly.

Verified locally on 1.98.1 with the same flags as CI:
cargo check --all-targets --all-features, cargo fmt --check, and
cargo check -p libsql --no-default-features for core/replication/remote.
@raphaelfeitoza
raphaelfeitoza force-pushed the fix/namespace-directory-ownership-89265 branch from 744ca2f to 09e4e6a Compare October 1, 2026 19:54
@raphaelfeitoza
raphaelfeitoza removed this pull request from stack #37 October 2, 2026 19:35
@raphaelfeitoza
raphaelfeitoza added this pull request to stack #40 October 2, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants